This invention relates to an image pre-processing sub-system for a computer-controlled visual feedback system intended for user-defined high speed machine control and object inspection applications.
Various image processing systems have been employed to translate an image obtained from a video camera to data which can be interpreted by one or more computers. Very computation intensive methods have been employed to make this translation. The trend is toward hardware support of this processing task to improve the response time. The fastest of such systems is too slow for most dynamic control operations, even when several parallel high speed microprocessors support the controlling computer. Insufficient support for linkage of boundary segments, detected by hardware or software support sub-systems, has left too much processing to the controlling computer. The result has been that image processing applications are slow. Control systems, depending upon visual feedback, are further limited to simple, slow dynamic operations. Consequently, a visual feedback control market is presently in its infancy. Product inspection apPlications have utilized the majority of image processing equipment developed to date. These systems are relatively expensive because of the associated computer and hardware costs. The inspection throughput rate is limited because of the long processing time required; this further adds to the inspection cost.
Most image processing systems conform to the traditional mathematical methodologies for translation of video camera output to usable data. These methodologies include filtering, thresholding, and processing of convolution patterns. Usually, the output of these processing operations is an enhanced set of data of equal or greater volume than the digitized video source data processed initially. These data are then further processed by hardware, software, or a combination, into boundary data recognizable by the controlling computer. The volume of processing operations and data handling operations is enormous. To speed the conclusion from these operations, parallel path processing techniques have been employed.
High resolution image processing has been a requirement for inspection operations. The operating speed must roughly increase with the square of the desired linear resolution, to process image data in a given time. Consequently, resolution limitations are presently resolved at the price of sacrificed operation speed.
Depth perception is presently achieved by extensive processing which typically accomplishes image ranging having poor accuracy. The long processing time limits machine control applications to very slow speeds.
An image pre-processing sub-system according to the present invention provides processed and linked image boundary data to a controlling computer of a system and provides electronic camera direction and magnification control capabilities to the computer to establish a high speed image evaluation system capable of high speed depth perception. The presently disclosed apparatus includes a video camera, which is controlled by an external system computer, and an image pre-processor, which processes video data to provide boundary linkage, texture, and shades-of-gray data to the computer. The computer is considered external to the sub-system since user supplied software provides the unique control response required by the user. The computer identifies the area of investigation, based upon the input data from the image pre-processor and the user supplied software computer control. It also provides electronic aiming controls to the camera. The disclosed embodiment illustrates data transfer, from the image pre-processor to the computer through interface circuitry, through a fiber optic link. The command path, from the computer interface to the image pre-processor, is also a fiber optic link. The integral interaction between the camera and the image pre-processor provides the high resolution processing and direction determining capability at real time processing speed. The extent of completeness of the processing allows one or more computers to receive this data and perform useful control tasks in real time.
The video camera is designed to provide output video data representing all or a selectable portion of the entire video image received by the camera. It is designed to scale the output data inversely proportional to the amount of reduction of the selected portion of the entire image. This permits the camera to output the same number of lines and the same time duration of line sweep time over the reduced portion size. Consequently, the smaller image portion is magnified and has greater line density and line spacing density, resulting in greater resolution. Therefore, as the image portion is reduced, the utilized resolution is increased. Reduction of the selected image portion can be obtained, with resulting resolution improvement, until the resolution limits of the camera video tube and supporting circuitry are reached. The center location of the reduced image portion can be positioned in both the horizontal and vertical directions. The camera has means to control the horizontal angle that the line sweep travels permitting the output image to be rotated. The camera responds from computer controls to reduce the image portion, to relocate the portion center, and to rotate the image. The resolution, of the step size of each of these parameters, is determined by the respective control signal resolution.
The image pre-processor provides the conversion from video data to a condensed usable form recognizable by a computer. The first step of the process is to digitize the video data. The data currently digitized are compared with data detected a few sample intervals earlier to determine if they exceed a computer controlled threshold. This process takes place at each sample interval and is accomplished in both the horizontal and vertical directions. When sufficient redundant samples indicate that a contrast exists, a boundary condition is determined. With the display area sub-divided into 4 .times. 4 (horizontal .times. vertical) sample interval spacings in units called cells, the four cell sides are evaluated for boundary detection. If an image boundary exists, a tangent line segment through the cell is identified. This tangent, an intersection of two tangents, or lack of boundary data (null) condition is encoded into an intersect code. The intersect code possesses information identifying the tangent segment, the segment's angle from horizontal, and the adjacent cells connecting. An indication of the side of the boundary containing greater light intensity is also provided. The horizontal component of contrast data is also used to provide a texture indication which is a count of the rate of occurrence of contrast detections per unit number of sample intervals. An average shades-of-gray indication for each cell is developed. An average image intensity is also developed for each field displayed which provides the computer data to regulate the video gain controls. Each of these data is temporarily stored, providing a delay which permits an ordered output of data with the sweep position (relating to the camera) identifying the data location and type.
The boundary detection processing determines boundaries without resorting to the mathematic methodologies commonly used by current technology. A small fraction of a field of data is stored at any one time. The propagation delay from the initial digitizing of data, at an area of the image, until an output results with encoded data of the area is only a few scan lines of the image as a result of the comparison technique. The comparison technique provides for noise filtering and boundary enhancement due to a majority logic processing employed to detect boundary conditions. The data management sorting and output control is accomplished by hardware to speed the process. The encoded data output to the computer is in condensed form. The output data is processed sufficiently to relieve the computer of time consuming boundary chaining tasks.